Training binary neural networks with knowledge transfer
نویسندگان
چکیده
منابع مشابه
Transfer Learning with Binary Neural Networks
Previous work has shown that it is possible to train deep neural networks with low precision weights and activations. In the extreme case it is even possible to constrain the network to binary values. The costly floating point multiplications are then reduced to fast logical operations. High end smart phones such as Google’s Pixel 2 and Apple’s iPhone X are already equipped with specialised har...
متن کاملKnowledge Transfer between Neural Networks
The goal of knowledge transfer is to take advantage of previous training experience to solve related but new tasks. This paper tackles the issue of transfer of knowledge between radial basis function neural networks. We present some preliminary work illustrating how a neural network trained on one task (the source) can be used to assist in the synthesis of a new but similar task (the target).
متن کاملConstructive Training Methods for feedforward Neural Networks with Binary weights
Quantization of the parameters of a Perceptron is a central problem in hardware implementation of neural networks using a numerical technology. A neural model with each weight limited to a small integer range will require little surface of silicon. Moreover, according to Occam's razor principle, better generalization abilities can be expected from a simpler computational model. The price to pay...
متن کاملBinaryConnect: Training Deep Neural Networks with binary weights during propagations
Deep Neural Networks (DNN) have achieved state-of-the-art results in a wide range of tasks, with the best results obtained with large training sets and large models. In the past, GPUs enabled these breakthroughs because of their greater computational speed. In the future, faster computation at both training and test time is likely to be crucial for further progress and for consumer applications...
متن کاملKnowledge Transfer Pre-training
Pre-training is crucial for learning deep neural networks. Most of existing pre-training methods train simple models (e.g., restricted Boltzmann machines) and then stack them layer by layer to form the deep structure. This layerwise pre-training has found strong theoretical foundation and broad empirical support. However, it is not easy to employ such method to pre-train models without a clear ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Neurocomputing
سال: 2020
ISSN: 0925-2312
DOI: 10.1016/j.neucom.2018.09.103